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Abstract 

The  DoD  requires  insight  into  the  stress  that  reduced  acquisition  spending  is  placing  on  the 
industrial  base.  Implementing  a  reliable  and  validated  methodology  would  facilitate  DoD-wide 
mitigation  actions  designed  to  preserve  or  enhance  key  industrial  base  capabilities.  The 
Office  of  the  Deputy  Assistant  Secretary  of  Defense  for  Manufacturing  and  Industrial  Base 
Policy  has  developed  and  tested  a  methodology  for  measuring  the  criticality  of  key 
capabilities  (products,  components,  technologies)  and  their  fragility  within  the  industrial  base 
(vendors  and  markets).  The  methodology  is  designed  to  be  used  across  services  and 
programs.  Pilot  testing  conducted  in  2013  of  selected  DoD  programs  and  sectors  applied  this 
methodology  to  measure  nine  constructs  of  criticality  and  six  constructs  of  fragility.  The  pilots 
assessed  sectors  and  programs  in  diverse  life-cycle  phases,  for  example,  development, 
production,  and  sustainment.  Data  collected  from  surveys,  datasets,  reports,  and  industry 
subject  matter  experts  were  used  to  assess  the  measures  for  capability/supplier  pairs. 
Assessments  were  plotted  in  a  fragility  and  criticality  matrix  to  inform  mitigation  strategies. 
Empirical  and  statistical  analyses  indicate  the  methodology  yields  useful  objective  measures 
of  risks  in  the  industrial  base.  Results  of  the  pilot  tests  were  used  for  mitigation  action  and 
improved  the  methodology  for  future  assessments. 

Introduction 


The  character  of  the  defense  industry  has  changed  significantly.  The  large 
prime  contractors  and  major  subcontractors  are  no  longer  stand-alone 
organizations  devoted  primarily  to  defense  business.  The  companies  have 
become  elements  of  large  multi-product,  multi-market  organizations,  and 
must  compete  internally  for  the  limited  capital  that  is  available.  ...  The  decline 
in  engineering  and  scientific  graduates  is  well  documented,  but  perhaps  less 
visible  in  the  current  increasing  shortage  of  skilled  production  workers, 
electronics  technicians,  tool  and  die  makers.  ...  The  problems  of  the  primes 
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and  large  subcontractors  are  very  often  intensified  in  smaller  subcontractors 
and  suppliers.  (Defense  Science  Board,  1981,  pp.  7,  9,  18) 

The  findings  from  a  Defense  Science  Board  (DSB)  Summer  Study  Panel  on 
Industrial  Responsiveness  remains  as  relevant  today  as  it  was  over  30  years  ago.  In  2008, 
the  DSB  Task  Force  on  Defense  Industrial  Structure  for  Transformation  concluded  that  the 
end  of  the  Cold  War  once  again  had  a  significant  impact  on  the  U.S.  industrial  base  due  to 
drastic  reductions  in  the  defense  budget.  Their  report  echoed  warnings  about  the  health  of 
companies  that  support  the  DoD,  the  lack  of  independent  research,  and  loss  of  scientists 
and  engineers  in  the  defense  industry,  and  urged  the  DoD  to  develop  incentives  to  achieve 
“a  balance  between  innovation  that  delivers  superior  capability  and  increased  production 
quantities  of  existing  designs”  (DSB,  2008). 

While  the  national  security  environment  and  issues  are  different  in  the  two  reports, 
they  are  consistent  in  their  expression  of  concern  that  the  industrial  base  may  not  be  able  to 
meet  the  future  DoD  needs  in  equipping  and  sustaining  military  forces.  And  in  2014,  the 
concern  remains  as  the  Department  must  make  important  decisions  about  its  investment 
priorities  as  the  procurement  budget  is  reduced  by  40%  from  2010  levels. 

However,  cycles  in  budgets  are  not  new  to  the  DoD.  In  order  to  understand  and 
mitigate  the  impacts  to  programs  from  changes  in  the  industrial  base  and  vice  versa,  the 
DoD  published  a  Directive  with  a  handbook  in  1996  formalizing  the  assessment  of  defense 
industrial  capabilities  on  a  case-by-case  basis  (DoDD  5000. 60H,  1996).  Once  an  area  of 
concern  is  identified,  the  handbook  provides  a  framework  to  determine  the  need  for 
government  action  to  preserve  industrial  capabilities  vital  to  national  security.  The  framework 
is  useful,  but  reactive  and  program-centric. 

The  current  period  of  lower  DoD  acquisition  spending  is  expected  to  place  stress  on 
the  industrial  base,  particularly  those  parts  that  are  dependent  on  defense  spending.  Budget 
cuts  result  in  reduced  levels  of  procurement  for  a  program,  delays  in  starting  new  programs, 
and  cancelation  of  existing  programs.  While  primes  manage  their  individual  supply  chains 
effectively,  the  cumulative  effect  of  multiple  program  procurement  decisions  can  have 
unintended  consequences  on  vital  capabilities.  Accordingly,  the  DoD  requires  proactive 
insight  into  the  impacts  of  acquisition  decisions  made  today  on  the  industrial  capabilities  it 
requires  now  and  in  the  future.  Building  on  the  existing  framework,  the  Office  of  the  Deputy 
Assistant  Secretary  of  Defense  for  Manufacturing  and  Industrial  Base  Policy 
(ODASD[MIBP])  developed  a  methodology  that  could  be  used  proactively,  across  Services 
and  industrial  sectors,  that  is  rigorous,  repeatable,  and  transparent.  This  paper  describes 
the  results  of  a  pilot  effort  conducted  through  summer  2013,  the  methodology,  and  next 
steps. 

Budget  Cycles  and  the  Department  of  Defense 

Defense  investment  outlays,  including  funding  for  procurement  and  research, 
development,  test  and  evaluation  (RDT&E),  have  been  cyclic  since  at  least  World  War  II 
(see  Figure  1).  Budget  swings  have  significant  consequences  for  the  industrial  base  as  they 
try  to  make  business  decisions  around  the  volatility.  During  the  upswings,  defense 
contractors  and  their  vendors  must  acquire  resources,  which  are  sometimes  limited  and 
specialized,  to  address  their  schedule  and  performance  requirements.  During  the 
downswings,  those  same  contractors  must  look  into  their  crystal  balls  and  make  strategic 
decisions  on  how  much  of  that  capability  they  can  afford  to  maintain  in  anticipation  of  the 
next  upswing,  or  decide  that  the  risk  is  not  worth  the  reward  and  exit  the  defense  market. 


ACQUISITION  RESEARCH  PROGRAM: 
CREATING  SYNERGY  FOR  INFORMED  CHANGE 


-  157- 


DoD  Investment  Outlays 


oo«H^r''“OcovDO'i<NiooorHTfr^omvomfN‘^ooi“«Tjr','> 

^munm^ijp^pvpr^rvr^oocoQOcnmcncnQOQ^HTHtH 

OiO'>OlO>u101<7lo>Cl^GlolG^tr»cn^ulChOOOOOO 
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Figure  1.  DoD  Investment  Outlays,  1948-2017,  Constant  2014  Dollars 

(Source:  National  Defense  Budget  Estimates  for  FY  2014,  May  2014,  Tables  6-11) 

Research  suggests  that  DoD  budget  upswings  have  very  limited  effects  on 
technological  advances  despite  the  influx  of  dollars  (Mowery,  2013).  Instead  investments  in 
research  and  innovation  during  mobilization  focus  on  bringing  near-term,  largely  incremental 
technology  to  the  solider.  As  a  result,  investment  decisions  made  during  downswings  are 
vitally  important  to  ensuring  that  the  industrial  base  that  supports  the  DoD  will  be  able  to 
provide  the  capabilities  needed  at  the  right  time  and  quantity  to  maintain  superiority  over 
adversaries. 

Reduced  DoD  procurement  affects  prime  vendor  decisions  on  how  to  absorb  cuts, 
and  their  decisions  flow  down  to  the  sub-tier  suppliers.  The  fundamental  questions  from  an 
industrial  base  perspective  in  deciding  what  and  where  to  make  cuts  are:  Will  a  capability 
that  is  needed  to  support  the  warfighter  be  available  during  a  downturn  and  at  the  next 
upturn?  Will  the  industrial  base  support  the  next  generation  of  innovation? 

Characterizing  Risk  in  the  Defense  Industrial  Base 

The  DoD  faces  two  kinds  of  risk  in  the  industrial  base — voluntary  and  involuntary. 
Involuntary  risk  is  incurred  from  external  influences  such  as  the  Defense  procurement 
budget,  congressional  mandates,  and  emerging  national  security  threats.  Each  of  these 
influences  impact  DoD  decisions  on  amount  and  type  of  activity  that  is  distributed  among  the 
predominant  factors  of  research  and  development,  acquisition,  and  sustainment. 

Voluntary  risk  arises  from  the  consequences  of  decisions  within  the  Department’s 
sphere  of  influence.  This  includes  decisions  about  when  and  how  the  Department  will  rely 
on  market  forces  to  create,  shape,  and  sustain  industrial  and  technological  capabilities,  as 
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well  as  how  it  will  intervene  when  absolutely  necessary  to  create  or  sustain  essential 
industrial  capabilities. 

During  the  Korean  War,  DoD  investment  outlays  comprised  over  5%  of  total  gross 
domestic  product  (GDP),  sustaining  a  strong  symbiotic  relationship  with  U.S.  industry  of 
earlier  eras.  The  influence  of  defense  investment  outlays  has  dropped  since  the  late  1980s 
to  less  than  2%  of  GDP,  with  a  concentration  of  certain  industries,  globalization  of  markets, 
and  increased  export  markets.  The  simple  fact  of  having  a  private  sector  industrial  base, 
frequently  international,  that  is  largely  outside  of  the  DoD’s  control  reduces  the  Departments 
options  for  risk  mitigation.  However,  that  limitation  is  only  a  minor  deterrent  to  action. 

Before  we  can  implement  any  mitigation  actions,  we  must  first  identify  the  action  as  a 
solution  to  a  known  industrial  base  deficiency.  Before  an  industrial  base  deficiency  can  be 
known,  we  must  be  able  to  compare  defense  requirements  to  industrial  base  capabilities. 
Before  we  can  compare  industrial  base  capabilities  to  requirements,  we  must  have  visibility 
into  capabilities  of  individual  product  and  service  providers  and  their  supplier  networks. 

In  2008,  the  Government  Accountability  Office  reported  that  the  DoD’s  efforts  to 
monitor  its  supplier  base  lacked  a  Department-wide  framework  and  consistent  approach. 

The  report  noted  that  DoD  monitoring  efforts  generally  respond  to  individual  program 
supplier-base  concerns  or  are  broader  assessments  of  selected  sectors;  use  an  informal 
approach  to  identify  supplier-base  concerns,  often  relying  on  the  Military  Services,  program 
offices,  or  prime  contractors  to  identify  and  report  these  concerns,  including  gaps  or 
potential  gaps;  and,  since  no  requirement  for  when  to  report  such  gaps  to  higher-level 
offices  exist,  knowledge  of  defense  supplier-base  gaps  across  the  DoD  may  be  limited. 

In  2011,  ODASD(MIBP)  was  tasked  with  developing  a  forward-leaning  approach  that 
could  identify  the  cumulative  effect  on  vital  capabilities  of  procurement  decisions  across 
programs  and  Services.  We  built  on  the  existing  1996  framework  to  develop  a  methodology 
that  could  be  used  proactively,  across  Services  and  industrial  sectors,  that  is  rigorous, 
repeatable,  and  transparent.  The  process  became  known  as  the  Sector-by-sector,  Tier-by- 
tier  Fragility  and  Criticality  (FaC)  assessment  process,  or  FaC  for  short. 

Methodology 

The  DoD  tested  the  FAC  Assessment  Process  by  completing  10  pilot  FAC 
assessments  in  2013  that  included  a  mix  of  sectors  (space,  missiles,  military  satellite 
communications  terminals,  and  focal  plane  arrays),  programs  (WIN-T  Inc.  1 ,  F-18,  F-22, 

Gray  Eagle),  and  skill  areas  (organic  depot,  vertical  lift).  From  these  pilots,  the  DoD  refined 
the  process  and  the  resulting  high-level  overview,  presented  in  Figure  2.  The  process  is 
designed  to  be  iterative  and  provide  continuous  expansion  of  the  DoD’s  insight  into 
industrial  base  capabilities  and  constraints. 

The  first  activity  in  the  FaC  assessment  process  is  to  select  the  assessment  subject 
and  scope.  The  assessment  generally  begins  by  choosing  an  industrial  base  sector  or 
subsector  within  that  sector.  However,  as  additional  insight  is  acquired,  future  iterations  may 
focus  on  more  limited  technology  or  commodity  areas.  Selection  of  a  program  or  sector  for  a 
FaC  assessment  is  based  on  leadership  priorities,  industrial  base  analysis,  and  the  results 
of  prior  industrial  base  assessments. 
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Figure  2.  FaC  Assessment  Activities,  Actions,  and  Outcomes 

A  functional-  or  component-based  taxonomy  of  the  sector  is  used  to  scope  the  FaC 
Assessment,  indicating  the  focus  (and  non-focus)  of  the  assessment.  Sectors,  subsectors, 
and  programs  intersect  in  a  number  of  ways,  and  the  taxonomy  provides  the  “boundary”  for 
what  is  in  or  out  of  scope  for  a  particular  assessment  and  highlights  where  future  efforts 
might  focus.  The  FaC  is  designed  as  an  iterative  process,  and  the  taxonomy  is  instrumental 
to  identify  the  sub-tier  markets  that  comprise  the  sector,  and  to  decide  when  to  go  broader 
or  deeper  in  a  subsequent  iteration.  A  preliminary  schedule  and  initial  analysis  prepares  the 
working  group  members  on  their  role  and  expected  assessment  outcomes.  This  helps  to 
identify  functional  areas  that  may  also  impact  other  sectors,  allowing  for  data  re-use. 

A  key  element  of  the  process  is  to  create  the  FaC  Integrated  Product  Team  (IPT). 

The  IPT  is  a  mature  and  well-established  mechanism  for  having  cross-organizational  and 
cross-disciplinary  dialog  about  an  issue  area.  A  FaC  sponsor  determines  when  there  is 
sufficient  justification  to  proceed  with  a  FaC  assessment,  and  acts  as,  or  identifies,  a  FaC 
IPT  chair.  The  chair  is  responsible  for  establishing  the  IPT,  setting  and  overseeing  the  FaC 
assessment.  The  IPT  chair  hosts  a  kickoff  meeting  where  the  IPT  members  are  briefed  on 
the  selected  sector  and  the  information  collected  to  date.  The  IPT  members  are  trained  on 
the  FaC  assessment  process  to  make  sure  that  the  results  of  this  assessment  are  consistent 
with  other  assessments  being  conducted  by  other  groups.  Due  to  the  iterative  nature  of  the 
FaC,  the  IPT  must  be  equally  flexible  to  ensure  that  the  requisite  expertise  is  available  for 
the  assessment. 

The  second  activity  in  the  process  is  to  search  for  data  and  filter  out  non-industrial 
base  issues  to  support  industrial  base  assessment.  In  today’s  resource  constrained 
environment,  the  analyst  cannot  afford  to  conduct  an  open,  unbounded  search  for 
information.  Once  the  assessment  scope  is  selected,  industrial  base  analysts  will  evaluate 
available  data  sources  for  potential  inclusion  in  the  FaC  assessment.  Specific  program  or 
sector  and  supplier  information  included  in  existing  databases,  tools,  programs,  etc.  is 
identified  through  the  FaC  criteria  lens.  Care  is  taken  to  ensure  a  transparent  link  to  all  data 
sources,  and  to  share  data  sources  among  the  FaC  assessments.  When  the  analyst  finds 
deficiencies  in  the  available  information,  they  may  contact  subject  matter  experts  (SMEs) 
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knowledgeable  in  relevant  technologies  or  the  acquisition  supply  chain  to  augment  the 
knowledge  base. 

A  DoD  platform  can  have  thousands  of  parts  and  associated  vendors  and  an 
industrial  base  sector  has  even  more — so  many  that  it  would  be  impractical  to  evaluate  all  of 
them  in  any  single  assessment.  Accordingly,  before  conducting  a  FaC  assessment,  the  IPT 
applies  a  set  of  filters  to  arrive  at  the  target  set  of  capabilities  and  vendors.  The  filtering 
activity  is  essential  to  the  FaC  process:  filtering  rids  the  assessment  of  non-industrial  base 
issues,  and  it  protects  against  data  overload  by  focusing  the  efforts  of  the  IPT  on  areas  of 
higher  probability  of  risk. 

While  some  filters  were  used  in  each  IPT,  each  IPT  also  chose  additional  filters 
tailored  to  the  sector  or  program  under  investigation.  Since  the  focus  of  the  assessment  is  to 
identify  risks  in  the  defense  industrial  base,  a  crucial  task  is  to  weed  out  risks  that,  while 
important,  are  not  important  to  the  industrial  base.  Specifically,  program-related  supply  chain 
issues  do  not  necessarily  impact  the  industrial  base.  Accordingly,  one  of  the  screens  applied 
to  each  capability  is  whether  it  is  relevant  to  many  platforms  and  Services.  If  the  answer  is 
no,  there  may  be  risk  in  the  supply  chain  for  the  capability,  but  it  is  not  considered  a  risk  in 
the  industrial  base  since  mitigation  efforts  are  appropriately  handled  by  the  prime  vendor  or 
program  office  affected  by  the  issue. 

A  primary  objective  of  the  FaC  pilots  was  to  achieve  consistent  application  of 
industrial  base  assessment  techniques  across  the  assessment  teams.  To  that  end,  the 
assessment  community  developed  a  FaC  Assessment  Candidates  List  (FAC-List).  The 
FAC-List  is  a  standard  template  used  to  collect  the  product  and  supplier  identification 
information  and  record  filtering  decisions.  While  the  FAC-List  template  contains  default 
filters,  each  FaC  IPT  is  able  to  add  other  filters  as  deemed  appropriate.  Some  filters  apply  to 
all  FaC’s,  for  example,  identifying  capabilities  associated  with  single  sources  of  supply,  while 
others  are  specific  to  the  sector  under  investigation.  The  maturity  of  a  capability  may  have  a 
significant  impact  on  the  types  of  filters.  A  product  or  technology  in  development  focuses 
more  on  attributes  of  new  product  or  service  technologies  and  prototype  integration,  while 
established  products  and  technologies  may  focus  on  issues  of  obsolescence. 

The  result  of  the  data  collection  and  filtering  activity  is  a  winnowed  down  set  of 
“capability  and  supplier”  pairs  that  are  moved  forward  for  the  formal  FaC  assessment.  In  the 
assessments,  a  capability  required  from  the  industrial  base  is  defined  as  a  technology,  part, 
component  or  product.  A  supplier  is  defined  as  the  current  provider(s)  of  that  capability.  A 
capability,  then,  might  be  as  small  as  an  individual  part  or  as  large  as  an  integrated 
subcomponent,  each  capability  then  paired  with  its  current  suppliers,  and  each 
capability/supplier  pair  assessed  separately.  Accordingly,  the  same  capability  may  be 
associated  with  multiple  suppliers,  and  a  single  supplier  may  be  associated  with  multiple 
capabilities. 

The  third  activity  is  to  conduct  the  FaC  assessment.  The  heart  of  the  assessment 
process  is  the  set  of  criticality  and  fragility  criteria  that  serves  as  indicators  of  potential 
industrial  base-related  risk.  Criticality,  from  an  industrial  base  perspective,  consists  of 
indicators  to  identify  when  a  capability  would  be  difficult  to  replace  if  it  was  lost  or  disrupted. 
Fragility  indicators  focus  on  the  robustness  of  current  suppliers  of  a  capability  and  the 
availability  of  potential  firms  in  the  current  marketplace.  Table  1  shows  the  criticality  and 
fragility  factors  that  the  DoD  initially  selected  for  the  pilot  assessments. 
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Table  1.  Fragility  and  Criticality  Factors  During  2013  Pilot  Testing 


Criticalitv:  Characteristics  that  it 
=  technology,  part,  component, 

ake  a  specific  Capability  difficult  to  replace  if  disrupted  (Capability 
product 

Defense  unique  capability 

To  what  degree  is  the  market  for  this  capability  commercial? 

Relevance  to  DoD 

How  pervasive  is  this  capability  across  the  DoD? 

Skilled  labor  requirements  for 
the  capability 

To  what  degree  are  specialized  skills  needed  and  available  to 
integrate,  manufacture  or  maintain  this  capability? 

Design-intensive  activity 

To  what  degree  is  defense-specific  knowledge  required  to 
reproduce  this  capability,  an  alternative,  or  the  next  generation 
design? 

Level  of  future  demand 

What  is  the  expected  global,  commercial,  military,  and  other 
government  demand  over  the  next  5  years? 

Reconstitution  cost 

What  is  the  impact  on  the  DoD  to  restore  this  capability  if  it  is 
lost? 

Availability  of  alternatives 

To  what  degree  are  cost,  time,  and  performance-effective 
alternatives  available  to  meet  DoD  needs? 

Geo-political  climate 

Where  is  the  market  for  this  capability  and  what  is  the  degree  of 
foreign  dependency  in  terms  of  alternative  suppliers? 

Long  lead  item 

What  is  the  impact  on  the  DoD  from  the  lead  time  to  obtain  the 
capability  from  the  current  market? 

Fraqilitv:  Characteristics  that  make  a  specific  Capability  likelv  to  be  disrupted  (Will  the  Department 
receive  what  it  needs,  when  it  needs  it?) 

Financial  outlook  (Current 
provider) 

What  is  the  risk  of  this  facility  going  out  of  business  or  exiting  the 
market  for  this  capability? 

DoD  dependence  (Current 
provider) 

What  percentage  of  total  sales  for  this  facility  are  from  DoD 
contracts? 

Firms  in  sector  (Existing 
market) 

How  many  firms  currently  participate  in  this  firm’s  market  for  this 
capability? 

Impact  on  firms  cost  from 
variation  in  output 

What  is  the  estimated  impact  from  a  50%  change  in  DoD 
procurement  on  the  unit  cost? 

Production  minimum 
sustaining  rate 

What  are  the  production  levels  of  the  firm  relative  to  MSR? 

Diversity  of  firm’s  earnings 
from  program  elements 

How  many  distinct  programs,  products,  or  service  groups  create 
revenue  for  the  firm? 

The  information  required  to  assess  FaC  criteria  in  combination  with  demographic  and 
economic  data  of  the  commercial  organizations  permits  industrial  base  analysts  to  sort  risks 
based  on  whether  a  given  risk  is  rooted  in  broad  industrial  base  issues  or  is  unique  to  a 
particular  capability.  Armed  with  the  filtered  list  of  target  capabilities  to  assess,  the  IPT 
members  hold  an  assessment  workshop  to  complete  a  FaC  assessment  using  the  fragility 
and  criticality  assessment  criteria  and  rating  scales.  During  the  workshop,  SMEs  evaluate 
the  criticality  and  fragility  factors  for  each  capability.  This  assessment  process  leads  to  a 
FaC  Assessment  Risk  Matrix,  identifying  the  most  critical  capabilities  and  fragile  suppliers. 
The  high-risk  capabilities  become  the  subject  for  further  investigation  and  validation.  SME 
recommendations  for  areas  that  require  further  investigation,  along  with  any  risk  mitigation 
suggestions,  are  some  of  the  workshop  results  and  a  basis  for  follow-up  actions. 

In  the  pilot  phase,  a  facilitator  guided  the  in-person  SME’s  through  the  scoring 
exercise.  The  IPT  members  first  assessed  each  capability,  assigning  a  score  on  a  5-point 
scale  for  each  criticality  measure.  When  SME’s  were  unanimous  on  a  rating,  the  score  was 
recorded  on  the  matrix.  When  SME’s  differed,  a  discussion  ensued,  with  comments 
captured.  Often  the  SME’s  agreed  on  a  rating  after  sufficient  dialogue.  However,  when  an 
agreement  could  not  be  achieved,  the  more  conservative,  higher  risk  rating  was  recorded 
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and  duly  commented,  and  the  process  continued  to  the  next  factor,  and  so  on,  until  each 
factor  and  capability  was  assessed.  After  rating  each  capability,  the  unweighted  scores  were 
averaged  and  plotted  on  a  5-point  scale.  As  the  pilots  progressed,  the  FaC  assessments 
began  to  focus  on  the  capabilities  with  the  highest  criticality  ratings  before  scoring  fragility 
based  on  the  assumption  that  if  a  capability  was  not  critical  from  an  industrial  point 
perspective,  there  was  (relatively)  less  concern  over  the  market  for  the  capability. 

After  establishing  the  criticality  ratings  for  each  capability,  the  IPT  members  focused 
on  the  fragility  factors.  The  process  for  rating  and  recording  each  factor  was  the  same  as 
described  above,  except  SME’s  now  focused  on  aspects  of  the  market  for  a  capability. 
Scoring  of  the  fragility  factors  turned  out  to  be  problematic  for  a  number  of  reasons.  First, 
the  IPT  members  who  had  deep  knowledge  about  the  capabilities  required  by  the  DoD, 
were  less  likely  to  know  financial  or  other  details  about  a  specific  firm.  As  discussed  in  the 
Results  section,  adjustments  made  to  the  fragility  measures  during  the  course  of  the  pilots 
improved  the  robustness  of  this  indicator,  which  will  be  tested  as  part  of  the  2014  FaC 
assessments. 

When  the  IPT  completes  the  individual  scores,  the  next  step  in  the  pilot  was  to  elicit 
weights.  The  discussion  and  application  of  weights  by  the  SMEs  was  important  to  gain 
insight  into  what  the  IPT  believed  to  be  important  determinants,  to  test  the  patterns  across 
the  10  pilots,  and  to  establish  the  impact  of  weighting  on  the  outcome. 

The  fourth  activity  is  to  validate  high-risk  industrial  issues  and  develop  mitigation 
strategies.  Figure  3  presents  a  visual  example  of  a  final  FaC  Assessment  Risk  Matrix 
generated  by  the  pilots.  The  dots  representing  capability/supplier  pairs  in  the  upper  right 
quadrant  of  the  chart  are  capabilities  that  represent  potential  risks  to  the  DoD  in  the 
industrial  base.  The  scoring  allows  IPT  members  to  focus  on  the  highest  risk  (red-orange) 
items  as  the  most  critical  and  fragile  elements,  which  are  also  mapped  back  onto  the 
taxonomy  to  reveal  whether  specific  subcomponents  contain  multiple  risks. 
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Assessments  Provide  Guidance  for  Action 
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Figure  3.  Fragility  and  Criticality  Risk  Matrix 

The  FaC  assessment  process  is  intended  to  be  a  rapid  filter  that  allows  more 
efficient  focusing  of  resources  on  the  highest  risk  areas.  Accordingly,  the  IPT  can  now  focus 
their  efforts  on  validating  a  few  dozen  capabilities  in  a  sector  to  determine  which  ones 
require  further  analysis,  continued  observation,  or  immediate  mitigation.  The  findings  and 
recommendations  are  presented  to  leadership  on  possible  mitigation  strategies. 

The  fourth  activity  brings  closure  to  the  preceding  three  activities  and  sets  the 
agenda  for  a  subsequent  iteration  of  the  FaC  analysis,  if  required.  A  subsequent  iteration 
could  involve  selecting  a  different  part  of  a  sector/subsector,  modifying  the  membership  of 
the  IPT  to  be  able  to  dig  deeper  into  the  sector  under  investigation. 

FaC  Results 

The  goal  of  the  FaC  assessment  pilots  was  to  test  a  DoD-wide  methodology  that 
could  be  used  proactively,  across  Services  and  industrial  sectors,  that  is  rigorous, 
repeatable,  and  transparent.  After  scoping  the  area  of  interest  through  use  of  a  sector 
taxonomy,  the  process  includes  a  structured  approach  to  rapidly  sift  through  information  to 
focus  on  potential  industrial  base-related  issues,  followed  by  a  facilitated  scoring  by  subject 
matter  experts  based  on  defined  criteria.  The  outcome  yields  a  manageable  set  of  issues  for 
mitigation.  In  practice,  the  pilots  provide  much  evidence  to  support  the  goals,  as  well  as 
areas  of  the  process  that  require  modification. 

One  of  the  most  salient  results  that  emerged  from  the  FaC  pilots  is  the  need  to  focus 
on  non-program  assessments.  An  early  assumption  was  that  FaC  assessments  should  be 
conducted  program-by-program  to  build  a  systematic  understanding  of  industrial  base 
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issues,  and  the  assessments  would  provide  input  into  program-level  budgeting.  Four  of  the 
assessments  were  conducted  at  the  program  level.  However,  program  managers  are  skilled 
at  managing  risks  in  their  respective  supply  chains,  and  limited  new  information  was 
gleaned  from  program-level  FaC  assessment.  On  the  other  hand,  the  FaC  pilots  conducted 
at  sector  levels  yielded  significant  and  new  information  about  potential  risks  across  multiple 
programs.  Indeed,  even  in  heavily  studied  sectors  such  as  space  or  missiles,  the  process 
helped  to  identify  new  risks,  as  well  as  point  the  way  to  potential  mitigation  based  on  the 
characteristics  of  the  risks.  The  challenge  with  presenting  risk  at  a  cross-program  and  cross- 
Service  level  is  that  budgeting  occurs  in  program  silos.  Accordingly,  results  of  the  2013  FaC 
pilots  were  taken  to  a  Deputy’s  Management  Action  Group1  on  the  Industrial  Base  in 
December  to  determine  how  to  resource  the  mitigation  actions. 

In  response  to  feedback  from  the  IPT  teams,  we  developed  several  tools  to  facilitate 
the  FaC  process.  As  mentioned  above,  a  structured  FaC-List  helped  to  capture  data  and 
comments,  and  the  filters  used  for  each  pilot.  The  process  of  filtering  industrial  base-related 
issues  evolved  over  the  first  several  FaC  pilots,  and  these  lessons  learned  were 
incorporated  into  subsequent  pilots.  The  FaC  Assessment  Results  Matrix  is  a  set  of  macro- 
enabled  worksheets  that  translate  the  individual  scores  onto  a  traditional  risk  matrix.  The 
tool  allows  the  IPT  members  to  visualize  the  results  in  real  time,  facilitating  discussion  of  the 
outcomes,  and  also  allows  members  to  do  various  “what  if”  scenarios  based  on  changes  to 
underlying  assumptions. 

A  key  result  of  the  pilots  is  validation  of  the  criteria  and  overall  assessment  results 
based  on  SME  knowledge.  At  the  conclusion  of  each  FaC  assessment,  the  SME’s  were 
asked  to  step  back  and  evaluate  whether  items  identified  as  high  risk,  were  indeed  high  risk, 
and  conversely,  if  those  items  rated  as  low  risk,  were  low  risk.  There  were  no  cases  of  a 
false  high  risk  identified.  In  a  few  cases,  an  IPT  member  would  articulate  a  low  risk  item 
should  be  rated  higher.  The  team  would  reevaluate  the  ratings  relative  to  the  definitions,  and 
the  facilitator  would  inquire  about  other  factors  that  might  be  missing.  The  activity  sometimes 
resulted  in  a  higher  rating  for  a  capability,  which  then  had  to  go  through  the  rigorous 
validation  process.  As  a  rule,  though,  the  SME’s  agreed  with  the  aggregate  ratings  and 
rankings,  which  was  viewed  as  an  important  outcome  of  the  pilot  effort. 

The  area  requiring  the  most  review  is  the  factor  rating  definitions.  The  initial  5-point 
rating  scale  was  designed  as  a  continuum  of  low  to  high  impact  for  a  particular  factor. 

During  the  pilots,  IPT  members  expressed  concern  around  some  of  the  scales  that  didn’t 
reflect  their  sector.  For  example,  the  original  scale  for  “Long  Lead  Time”  was  anchored  on 
actual  lead  times,  with  greater  than  two  years  posing  significant  risk.  However,  the  IPT  for 
some  sectors  indicated  that  lead  times  of  three  or  more  years  were  common,  and  programs 
factored  these  into  their  planning.  As  such,  the  fixed  scale  did  not  reflect  the  impact  of  lead 
time.  In  another  example,  the  factor  for  “Geo-political  climate”  was  used  as  both  a  criticality 
and  a  fragility  factor  under  the  assumption  that  items  with  only  foreign  sources  of  supply 
may  exhibit  risk  not  elsewhere  captured.  Half  of  the  pilots  included  a  modified  factor 
reflecting  this  concern  as  a  fragility  indicator.  The  pilots  also  revealed  that  the  firm-level 


1  The  Deputy’s  Management  Action  Group  (DMAG)  serves  as  the  Defense  Business  Systems  Management 
Committee  (DBSMC),  which  is  a  joint  committee  of  senior  leaders  responsible  for  executing  a  common  approach 
across  Departmental  processes. 
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information  was  not  available  on  a  consistent  level  except  for  financial  stability,  rendering 
some  indicators  largely  unused. 

Despite  the  difficulties  with  the  rating  scales,  like  any  good  pilot  testing,  the 
subsequent  IPT’s  continued  to  improve  as  the  definitions  became  sharper  and  easier  to  use. 
We  conducted  a  series  of  empirical  and  statistical  analyses  of  the  collective  ratings  to  further 
understand  the  results  of  the  pilots. 

Empirical  Evidence  From  Pilot  Tests:  A  repeatable  process  requires  repeatable 
results  under  a  variety  of  settings.  The  10  pilot  assessments  were  conducted  at  different 
levels  of  analysis — program,  sectors,  and  integrated  components.  The  question  thus 
arises — does  the  rating  structure  hold  up  under  any  level  of  analysis?  The  pilots  were 
designed  to  elicit  weighting  between  factors  to  understand  a  number  of  questions,  including, 
for  example,  whether  the  maturity  of  an  industry  segment  influenced  the  importance  of  some 
factors  over  others.  Consequently,  how  do  results  change  when  weights  from  one  pilot  are 
applied  to  others?  Finally,  the  fragility  factors  were  particularly  challenging  to  rate,  and 
evolved  over  the  course  of  the  pilot  tests.  What  is  the  impact  on  the  rankings  of  early  pilots 
when  certain  fragility  factors  are  excluded  from  the  assessment? 

During  the  FaC  process,  each  IPT  validated  the  rankings  and  results  of  their 
assessments.  We  used  this  observation  to  establish  internal  validity  of  an  assessment  since 
the  SMEs  were  satisfied  that  the  resulting  risk  matrix  accurately  portrayed  the  relative 
intensity  and  rank  order  of  industrial  base  risks  within  their  areas  of  expertise.  This 
fundamental  assumption  was  then  used  to  test  various  constructs.  The  first  step  was  to 
understand  the  impact  of  various  weighting  schema  on  outcomes. 

At  the  conclusion  of  the  rating  of  each  criticality  and  fragility  measure,  the  results 
were  displayed  on  the  risk  matrix  based  on  equal  weighting  of  each  measure.  Our  a  priori 
assumption  was  that  each  measure  did  not  contribute  equally,  yet  we  lacked  the  data  to 
assign  unequal  weights.  Thus,  one  of  the  tasks  of  the  IPT  facilitator  was  to  elicit  weights 
within  pilot  FaC.  The  facilitator  returned  to  the  list  of  criticality  measures  and  asked  a  series 
of  questions  aimed  at  determining  which  single  indicator  was  most  important,  recording  the 
responses  of  various  SMEs.  Then,  which  two  or  more  measures  were  most  important,  and 
so  on.  In  each  pilot,  the  same  four  criticality  indicators  were  ranked  as  most  important  to 
overall  criticality  rating:  defense  unique,  relevance  to  the  DoD,  availability  of  alternatives, 
and  reconstitution  cost.  Labor  skills  and  design  intensity  were  consistently  ranked  next 
highest  in  importance.  Future  demand,  geo-politics,  and  lead  times  were  ranked  lowest  or 
received  zero  weighting.  Although  IPTs  might  assign  different  actual  weights  to  a  measure, 
the  overall  consistency  of  which  factors  were  most  important  provided  insight  and  allowed 
us  to  test  for  stability  in  the  results  across  the  pilots. 

Weighting  the  measures  of  fragility  posed  some  challenges.  As  mentioned  earlier, 
the  SME’s  did  not  have  insight  into  the  level  of  detail  required  of  some  fragility  measures  for 
specific  suppliers.  Indeed,  information  on  minimum  sustainment  rates  or  product-cost 
changes  requires  intimate  knowledge  of  a  supplier  and,  further,  the  data  can  grow  stale 
more  rapidly  than  other  indicators.  Accordingly,  some  IPT’s  chose  to  ignore  these  measures. 
In  addition,  early  in  the  pilots,  one  of  the  IPT’s  chose  to  include  “foreign  dependence”  as  a 
fragility  indicator,  with  the  presence  of  only  foreign  sources  as  a  high  risk  and  two  or  more 
domestic  sources  as  a  low  risk.  Feedback  from  other  SMEs  spurred  us  to  include  the 
indicator  in  subsequent  pilots,  and  to  add  the  measure  into  completed  FaC  pilots.  Not 
surprisingly  perhaps,  when  weights  were  elicited  for  fragility  measures,  four  indicators  were 
highly  ranked  and  the  remaining  indicators  were  a  distant  second  or  not  weighted.  The  four 
fragility  measures  were:  financial  stability  of  the  firm,  DoD  dependence  of  the  firm,  number 
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of  firms  producing  in  the  market,  and  foreign  market  dependency.  As  with  the  criticality 
measures,  the  observed  regularity  across  the  pilots  of  a  core  set  of  fragility  indicators 
provided  understanding  of  the  process. 

The  FaC-Matrix  tool  was  designed  to  allow  real-time  changes  of  the  weights  for  each 
measure  to  test  the  impact  of  the  weights  within  each  pilot.  The  second  empirical  step  was 
to  combine  together  like  assessments — programs  with  other  programs,  sectors  with  other 
sectors — to  observe  for  anomalies  in  the  ratings.  Including  the  nine  criticality  measures  and 
four  fragility  measures  described  above,  the  result  of  this  exercise  revealed  that  all  risk 
rankings  in  the  combined  assessments  remained  stable.  The  final  step  was  to  combine  all 
assessments  into  a  single  FaC  matrix  to  observe  for  differences  across  the  different  levels  of 
analysis — program  or  sectors.  Again,  the  ratings  in  the  combined  assessments  remained 
stable.  We  found  no  instances  where  items  rated  as  high  risk  would  fall  out  of  the  upper 
quadrant,  nor  low  risk  items  rise  into  the  higher  quadrant. 

Statistical  Factor  Analysis  of  Criticality  Factors  From  Pilot  Tests:  Armed  with 
evidence  of  consistency  across  the  pilot  FaC  assessments,  we  next  performed  an 
exploratory  factor  analysis  of  the  criticality  measures.  The  goal  of  such  analysis  is  to  reduce 
the  number  of  dimensions — in  this  case,  criticality  attributes — by  grouping  them  into  factors 
that  vary  together  in  a  statistically  meaningful  fashion  (Gorsuch,  1997).  The  dimension 
reduction  aids  in  narrowing  the  search  for  meaningful  measures  by  identifying  which 
attributes  make  significant  contributions  to  the  variance  of  the  data  set. 

As  indicated  above,  the  pilot  study  data  on  the  fragility  indicators  held  up  under 
empirical  testing,  but  changes  over  the  course  of  the  pilots  yield  only  small  sets  of  data  for 
factor  analysis.  However,  data  collected  on  the  nine  criticality  attributes  were  consistent 
across  the  pilots.  The  data  were  diverse  in  that  they  were  drawn  from  subcomponents, 
broad  defense  industrial  sectors,  and  from  individual  programs  in  various  stages  of 
acquisition  or  sustainment.  Criticality  data  collected  from  the  pilot  studies  were  subjected  to 
a  principal  axis  factor  analysis  to  determine  which  of  the  attributes  made  significant 
contribution  to  the  observed  data  variance.  A  total  of  138  unique  and  complete  responses 
gathered  from  the  pilot  projects  were  available  for  analysis. 

Factor  Extraction.  The  data  were  processed  with  an  iMac  running  SPSS  Version  21 
for  the  OS  X  operating  system.  A  dimension  reduction  was  performed  on  the  data  set  using 
principal  axis  factoring.  Table  2  is  a  breakdown  of  the  extracted  factors  and  the 
corresponding  variance  explained  by  each  factor. 

Table  2.  Total  Variance  Explained 

Initial  Eigenvalues 


Total  %  of  Variance  Cumulative  % 


1 

2.753 

30.587 

30.587 

2 

1.393 

15.482 

46.069 

3 

1.110 

12.330 

58.399 

4 

.988 

10.972 

69.372 

5 

.796 

8.841 

78.213 

6 

.676 

7.506 

85.719 

7 

.466 

5.181 

90.901 

8 

.447 

4.970 

95.871 

9 

.372 

4.129 

100.000 

As  presented  in  the  top  row,  the  first  factor  accounts  for  about  30%  of  the  variance, 
with  almost  60%  of  the  variance  in  the  data  set  explained  by  factors  1, 2,  and  3,  indicating 
that  the  nine  criticality  attributes  (not  to  be  confused  with  the  nine  factors  extracted  by  the 
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analysis)  could  potentially  be  grouped  into  independent  factors,  with  each  factor  defined  by 
a  unifying  theme.  The  strongest  three  factors  were  retained  for  further  analysis,  based  on 
their  relative  strength  and  the  linear  decrease  in  contribution  of  the  remaining  three  factors. 

The  factor  solutions  were  rotated  using  a  Direct  Oblimin  transformation  and  loadings 
tabulated.  Tables  3  and  4  are  the  resulting  structure  and  factor  correlation  matrices, 
respectively.  As  presented  in  the  second  column  of  Table  3,  four  criticality  measures  make 
up  Factor  1,  as  demonstrated  by  absolute  value  levels  of  greater  than  0.4:  Defense  unique, 
labor  skills,  design  intensity,  and  reconstitution  cost.  Factor  2  is  driven  by  a  single  criticality 
attribute:  availability  of  alternative  capabilities,  which  is  also  cross-loaded  with  Factor  1. 
Factor  3  has  three  of  the  criticality  attributes  loading  at  greater  than  the  0.4  level  with  the 
almost  all  possessing  a  negative  polarity.  All  three  factors  display  independence  and  high 
orthogonality  as  illustrated  by  the  low  correlation  values  in  Table  4. 


Table  3.  Structure  Matrix 


Criticality 

Attribute 

1 

Factor 

2 

3 

Defense  Unique 

.787 

-.122 

-.112 

Relevance  to  DoD 

-.297 

.322 

.002 

Labor  Skills 

.668 

.028 

-.306 

Design  Intensity 

.663 

.209 

-.438 

Available  Alternatives 

.319 

.768 

-.266 

Future  Demand 

.323 

.103 

-.050 

Geo-Political  Climate 

-.004 

.185 

-.005 

Reconstitution  Cost 

.447 

.249 

-.664 

Lead  Time 

.075 

.017 

-.660 

Extraction  Method:  Principal  Axis  Factoring.  Rotation:  Oblimin  with  Kaiser  Normalization. 


Table  4.  Factor  Correlation  Matrix 


Factor 

1 

2 

3 

1 

1.000 

2 

.043 

1.000 

3 

-.250 

-.145 

1.000 

The  exploratory  nature  of  factor  analysis  calls  for  a  certain  level  of  interpretation.  It  is 
expected  that  candidate  attributes  and  constructs  will  not  contribute  to  explaining  the 
observed  variance  in  the  data.  The  dimension  reduction  and  interpretation  by  the  researcher 
are  tools  to  focus  on  the  attributes  that  best  explain  the  behavior  of  the  collected  data. 

The  components  of  Factor  1  display  a  unifying  theme  of  niche  capabilities.  A 
combined  high  risk  rating  would  describe  a  capability  that  has  low  commercial  applicability, 
requires  highly  skilled  labor,  demands  deep  design  knowledge  to  produce  an  alternative, 
and  would  have  a  high  impact  on  the  DoD  to  restore  the  capability  if  it  were  lost.  Factor  2,  as 
a  single  indicator,  does  not  require  reinterpretation,  but  when  viewed  through  the  lens  of  the 
first  factor,  the  combined  theme  suggests  characteristics  that  make  a  specific  product 
difficult  to  replace  if  disrupted,  which  is  the  description  we  employed  to  describe  critical 
capabilities  in  the  industrial  base.  The  third  factor  is  harder  to  interpret  since  criticality 
measures  were  not  reverse-coded  and,  as  noted,  the  loadings  possess  a  negative  polarity. 
Two  of  the  three  measures  are  included  in  Factor  1.  Feedback  received  during  the  pilots 
suggested  that  the  scale  for  Lead  Time  was  not  effective  to  capture  the  impact  of  long  lead 
times  on  a  capability.  Based  on  the  coherence  of  Factors  1  and  2,  and  uncertainty  about  the 
interpretation  of  Factor  3,  we  conducted  a  Cronbach’s  Alpha  statistic  to  assess  the  reliability 
of  the  construct  of  Factors  1  and  2.  The  reliability  statistic  of  0.74  based  on  five  items,  and 
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significant  item-total  statistics  provided  confidence  that  we  could  reduce  the  number  of 
Criticality  measures  from  nine  to  five. 

Analysis  Conclusions:  We  conducted  empirical  tests  on  the  combined  pilot  data 
using  the  five  criticality  measures  and  four  fragility  measures.  While  individual  scores  varied 
from  the  original  scores,  the  resulting  FaC  Risk  Matrix  was  unchanged  with  respect  to  the 
items  identified  as  low  risk,  moderate  risk,  and  high  risk.  Indeed,  the  reduced  number  of 
indicators  provided  more  separation  among  the  items,  that  is,  there  was  less  “clumping”  of 
capability/supplier  pairs  with  the  removal  of  non-differentiating  information. 

From  the  data  explorations  and  statistical  analysis,  we  determined  that  Criticality  as 
a  construct  is  measuring  those  capabilities  that  are  difficult  to  replace  if  lost.  Feedback 
during  the  pilot  FaCs  indicated  that  the  measure  for  “reconstitution  cost”  was  a  proxy  for 
other  factors,  including  a  missing  variable  related  to  the  cost  of  facility  and  equipment  to 
produce  a  capability.  In  addition  “cost”  was  not  precise  indicator  for  the  measure;  instead  the 
IPT  teams  assessed  the  impact  on  the  DoD  in  the  amount  of  time  to  restore  the  capability  if 
it  was  lost.  Building  on  these  findings,  we  tightened  the  definitions  of  criticality  and  fragility 
ratings  as  presented  in  Table  5  and  included  a  sixth  measure  of  criticality  to  account  for 
facility  and  equipment  requirements. 


Table  5.  Revised  Fragility  and  Criticality  Factors  for  2014  Assessments 


Criticality:  Characteristics  that 
technology,  part,  component, 

make  a  specific  Capability  difficult  to  replace  if  disrupted  (Capability  = 
product 

Defense  unique  capability 

To  what  degree  is  the  market  for  this  capability  commercial? 

Skilled  labor  requirements 
for  the  capability 

To  what  degree  are  specialized  skills  needed  and  available  to 
integrate,  manufacture  or  maintain  this  capability? 

Defense  design 
requirements 

To  what  degree  is  defense-specific  knowledge  required  to  reproduce 
this  capability,  an  alternative,  or  the  next  generation  design? 

Facility  &  equipment 
requirements 

Are  specialized  equipment  or  facilities  needed  to  integrate, 
manufacture,  or  maintain  this  capability? 

Reconstitution  time  for  the 
capability 

What  is  the  impact  on  the  DoD  in  time  to  restore  this  capability  if  it  is 
lost? 

Availability  of  alternatives 

To  what  degree  are  cost,  time,  and  performance-effective  alternatives 
available  to  meet  DoD  needs? 

Fraqilitv:  Characteristics  that  make  a  specific  Capability  likelv  to  be  disrupted  (Will  the  Department 
receive  what  it  needs,  when  it  needs  it?) 

Financial  outlook  (Current 
provider) 

What  is  the  risk  of  this  facility  going  out  of  business  or  exiting  the 
market  for  this  capability? 

DoD  sales  (Current 
provider) 

How  much  total  sales  for  this  facility  are  from  DoD  contracts? 

Firms  in  sector  (Existing 
market) 

How  many  firms  currently  participate  in  this  firm’s  market  for  this 
capability? 

Foreign  dependency 
(Existing  market) 

What  is  the  dependence  on  foreign  sources  for  this  capability? 

Next  Steps 

The  10  FaC  pilot  assessments  completed  in  2013  identified  important  risks  in  the 
industrial  base  and  provided  actionable  mitigation  strategies  to  DoD  leadership.  The  pilots 
also  provided  lessons  on  how  to  improve  the  overall  FaC  process,  reduce  ambiguity  in 
definitions,  and  provided  empirical  and  statistical  evidence  to  improve  the  constructs  for 
criticality  and  fragility.  In  2014,  we  will  continue  to  refine  the  process  during  the  next  round  of 
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assessments.  Although  we  expect  these  assessments  to  be  much  improved  as  we  reflect  on 
the  lessons  learned,  the  process  is  not  yet  fully  unified  and  consistent.  As  we  proceed  with 
the  2014  list  of  sectors,  subsectors,  and  systems,  we  will  also  be  pursuing  the  following 
process  improvement  activities: 


•  Access  to  Data:  The  2013  assessment  teams  were  often  unable  to  locate 
consistent  data  about  some  of  the  factors,  particularly  for  fragility  measures. 
Thus,  the  initial  evaluation  was  determined  by  SME  experience  or  group 
opinion,  and  subsequently  validated  if  the  capability  was  both  fragile  and 
critical.  In  2014,  MIBP  will  enhance  its  ability  to  acquire  reliable  data  by 
expanding  its  access  to  data,  including  fee-for-service  sources,  refining  its 
use  of  selective  survey  instruments,  and  acquiring  analysis  tools  that  improve 
the  ability  to  assemble,  evaluate,  and  visualize  industrial  base  information. 

•  Expanded  Use  of  Taxonomies:  A  taxonomy  for  each  FaC  assessment  is 
essential  to  the  process.  The  taxonomy  provides  a  scoping  mechanism  on 
the  front  end,  a  guide  during  mitigation,  and  a  map  for  future  iterations.  In 
2013,  the  taxonomy  was  not  always  used  throughout  the  entire  FaC  process. 
In  2014,  MIBP  will  work  to  standardize  the  taxonomy  format,  and  to 
incorporate  more  explicitly  the  taxonomy  into  each  activity  in  the  FaC. 

•  Standardized  Filters:  The  FaC  assessment  is  a  process  to  quickly  isolate 
the  risks  in  the  industrial  base  to  the  delivery  of  materiel  to  support  the 
warfighter.  The  2013  pilots  were  completed  in  six  months  or  less.  Our  aim  is 
to  complete  FaC’s  in  half  that  time  to  be  proactive  in  identifying  emerging 
issues,  and  agile  in  responding  to  existing  ones.  To  achieve  this  goal,  we 
need  to  institute  guides  for  prioritizing  the  capabilities  that  are  taken  forward 
to  the  FaC  Matrix  scoring  session.  The  pilots  provided  valuable  insight  into 
the  filters  that  can  help  triage  the  issues.  In  2014,  we  will  refine  and 
categorize  the  filters  to  reduce  the  amount  of  time  devoted  to  this  activity. 

•  Evaluation  of  Technology  Gaps:  Returning  to  the  definition  of  criticality, 
FaC  assessments  identify  specific  capabilities  that  are  difficult  to  replace  if 
disrupted.  We  believe  there  is  an  opportunity  to  contribute  to  the  body  of 
knowledge  of  how  to  evaluate  the  link  between  defense  procurement  and  the 
industrial  base.  In  short,  we  envision  risk  identified  through  FaC  as  a  “gap” 
between  Defense  and  commercial  capabilities — if  robust  commercial  demand 
exists,  then  the  capability  should  be  relatively  easy  to  replace  even  when 
Defense  procurement  investments  decline.  While  there  are  many  benefits  to 
overlapping  Defense  and  commercial  capabilities  (e.g.,  no  gap),  there  are 
drawbacks  as  well.  A  gap  between  Defense  and  commercial  capabilities  can 
be  good  for  national  security,  for  example,  when  the  Warfighter  is  equipped 
with  capabilities  that  are  not  available  to  our  adversaries.  In  2014,  we  will  use 
our  FaC  assessments  to  explicitly  examine  the  issue  of  which  gaps  provide 
benefits  to  the  Department,  and  which  ones  create  unnecessary  risks. 


Closing  Comments 


The  cascading  effects  on  [small,  highly  specialized  companies]  of  decisions 
that  the  Department  makes  at  the  overall  programmatic  level  must  be  better 
understood — to  ensure  that  critical  lower-tier  providers  have  the  capacity  to 
respond  to  these  decisions,  to  ensure  the  continued  supply  of  critical 
subcomponents  to  our  defense  industrial  base,  to  ensure  that  critical  skills 
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are  not  lost,  and  to  protect  our  national  security  from  the  risk  of  using 
compromised  supply  chains.  (2010  Quadrennial  Defense  Review,  p.  82) 

Budget  cycles  are  not  going  away.  ODASD(MIBP)  has  made  significant  headway 
into  instituting  a  rigorous,  repeatable,  and  transparent  process  to  identify  and  mitigate  the 
impact  of  DoD  investment  decisions  on  the  industrial  base.  The  pilot  tests  served  to  refine 
and  improve  the  process,  and  provided  defensible  results  used  to  inform  budget  decisions. 
The  FaC  methodology  promotes  a  proactive  industrial  base  perspective  by  not  assuming 
away  the  industrial  base,  by  conducting  regular  assessments  of  industrial  base  capability 
and  risk,  and  then  deciding  explicitly  when  to  accept  industrial  base  risk  and  when  to 
mitigate  that  risk.  In  other  words  we  just  don’t  want  to  let  risk  happen.  Over  the  next  several 
years,  the  iterative  conduct  of  FaC  assessments  will  provide  the  DoD  with  an  improved 
understanding  of  the  capability  and  risk  dynamic.  This,  in  turn,  will  help  ensure  that  the 
Department  can  protect  and  support  the  Warfighter  to  the  highest  possible  extent. 
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